Cognitive change detection system

ABSTRACT

A method of detecting a changed condition within a geographical space from a moving vehicle. Images of that geographic space are memorialized in conjunction with GPS coordinates together with its GPS coordinates. The same geographic space is traversed from the moving vehicle while accessing the route&#39;s GPS coordinates. The memorialized images are played back by coordinating the GPS data on a memorialized images with that of the traversed geographic space such that the memorialized images are viewed simultaneously with the geographic space being traversed. An observer traveling within the moving vehicle can compare the memorialized images with those being traversed in order to identify changed conditions.

RELATED APPLICATIONS

This application is a division of U.S. application Ser. No. 11/326,821,filed Jan. 6, 2006 now U.S. Pat. No. 7,751,051, which is based upon andclaims the benefit of priority from prior U.S. provisional applicationSer. No. 60/641,906 filed on Jan. 6, 2005. The entire contents of theseapplications are incorporated herein by reference in their entirety.

TECHNICAL FIELD

Video is typically recorded by time, for example, 30 frames per second.By recording frames according to distance traveled, a new database iscreated. This video or sequential image database incorporatesgeographically referenced images that are spatially related rather thantime related. By the selection of a distance between frames to identifywhich frames are to be acquired, a dramatically smaller database resultsthat is able to be searched more easily. This provides for adramatically reduced size in the database and is a technique of datacompression. Specific frames or data can be more easily retrieved bygeographic position that can be determined by Global PositionSatellites, Inertial systems or other electro mechanical sensors.Traditionally, forward looking video views have not been recorded bydistance traveled or geographic position. The present invention involvesa cognitive change detection system having particular utility formilitary or detection of a changed circumstance. This circumstance mayrepresent a hazard to a vehicle traveling upon the roadway, such as achange in the roadway, cracks in pavements, tunnels or bridges orroadside encroachments. In addition, changes in nearby objects such aspower lines, building facades, parked cars can be identified. The hazardcould consist, for example, of a munition which has been placed on theshoulder of a roadway and which is capable of being detonated when avehicle is within range of being impacted by detonation of the munition.By practicing the present invention, recognition of a changedcircumstance such as the placement of a munition as compared to apreviously memorialized record (database) of the roadway and surroundinglandscape can effectively be used to recognize a hazard enabling thevehicle operator to circumnavigate or render harmless the hazard ifnecessary. Other applications include nighttime driving to provide adaytime image of the current position and heading, a more precisenavigation to conform to a previously identified path and theenhancement of current imagery with previously acquired data or imagery.

It is also possible to use the invention to identify a current GPSposition by selecting the closest matching frame in the database with aknown GPS co-ordinate that was previously acquired.

BACKGROUND OF THE INVENTION

Often times, for example, military vehicles are called upon to travelroutes which are well known to hostile forces. As a consequence,munitions such as bombs and mines can be surreptitiously placed alongsuch a route for detonation at a critical time when such vehicles areproximate to such hazards. Unless a trained observer was so familiarwith a particular route that any change in the topography such as theplacement of a bomb or mine would readily come to the attention of sucha vehicle operator, it is difficult if not virtually impossible toforesee the hazard and thus avoid its consequences.

In the past, topography including roadways have been geomapped,principally by aircraft traveling above the to be mapped region from topdown views. However, in light of wobbling aerial camera platforms, lookangle limitations and lens distortions converting an aerial map of aregion to a surface based operator view, prior geomapping techniqueshave proven complex. Additional complications come from the mosaicingtechniques that are required to integrate frames along the route beingtraveled into a searchable database. Further, for a vehicle operator toidentify changes between geomapped imagery, it has been determined to becognitively most helpful to provide a split screen or some other type ofparallel tracking display of a geomapped route to the route currentlybeing navigated to enable rapid comparisons to be made between currentconditions and those which act as a standard upon which future actionsor detections would be based.

The hardware requires the simultaneous or near simultaneous display ofthe current image and the geographically referenced database image.

It is thus an object of the present invention to provide a ready meansof providing a vehicle operator sufficient information to enable theoperator to avoid a natural or manmade hazard or identify a specificchanges in nearby objects or their relative positions.

It is yet a further object of the present invention to provide a meansof providing baseline data for comparing a current route with a standardbaseline view of the same route taken previously for ready, real timecomparisons.

It is yet a further object of the present invention to provide means ofalerting the driver to roadside hazards by looking forward in thedatabase to what is over a hill or around a bend.

It is yet a further objective of the present invention to integrateinformation from the inspection of roadways or other roadside conditionsin a format which highlights such conditions and hazards and whichpresents them to a vehicle operator in a timely fashion to provide theoperator the opportunity to take effective remedial action.

It is a further object to provide daytime views given a GPS location andheading.

It is a further object to present to a driver a behind the vehicle viewbased on previous views with the vehicle perimeter represented on screenso that the current relationship of the wheels to hazards can be viewedand is approximately accurate to the current vehicle position.

It is a further object to provide a means to reduce the number of framesand size of a database by recording imagery by distance separationrather than time. This was usually done by recording all frames in asequence done at 30 frames per second and tagging the GPS position onall thirty frames. Although the invention can be practiced with all 30frames, the preferred embodiment is to record a frame after the camerahas moved a specified distance to reduce the bandwidth and storagecapacity required.

It is further object to utilize other forms of imagery or data such asladar, radar, sonar, magnetics, multi-spectral, audio, computerrenderings from depth maps or wireframes or other forms of data such assound that can be discretely acquired by geoposition. These alternativesources of data can be mixed such as thermal with daytime video,magnetics with video, multi-spectral with rendered imagery.

It is a further object to identify a current GPS position by selectingthe closest matching frame in the database with a known GPS co-ordinatethat was previously acquired.

These and further objects will be readily apparent when considering thefollowing disclosure.

SUMMARY OF THE INVENTION

The present invention involves a method of detecting a changed conditionwithin a geographic space from a moving vehicle. The method comprisescapturing and memorializing images of the geographic space inconjunction with GPS/geographic coordinates associated with saidgeographical space. The geographical space is traversed from said movingvehicle while accessing GPS or other geographic coordinates. Thememorialized images are accessed and played back by coordinatinggeographic coordinate data on said memorialized images with thetraversing of said geographic space such that said memorialized imagesbeing viewed are of the same geographical space being traversed (i.e.the position of the camera now and in the past are within the samegeographic coordinates and headings +/− some tolerance). Bothmemorialized images and the images of the traversed geographic space arepresented to an observer enabling the observer to make a directcomparison of the memorialized images and images of the geographic spacebeing traversed to the observer. Ideally, the memorialized images arecreated by employing a video camera which can also be used to presentthe geographic space being traversed. Alternatively, other imagecapturing devices can be employed such as infrared cameras, sonar, orsensor data such as magnetic or sound data that can be graphicallyrepresented on a screen. To present the most direct comparison, thecamera presenting the traversed images from the moving vehicle should beplaced in approximately the same location and heading as that of thecamera employed to capture the memorialized images with a similar fieldof view. Fields of view can be better matched optically or by electronicscaling. Image stabilization techniques can also be used to improve therelative registration between images. Both images can be presented upona screen, such as a split screen in which the traversed images arepresented above and the memorialized images are presented below.Alternative displays such as sequential presentations, alternating,super-imposition or keying and matting techniques can also be used. ByGPS coordination, the images of the same geography are presented to anobserver simultaneously enabling the observer to quickly and intuitivelyrecognize any changed conditions in the roadway. Further, although thememorialized images and traversed images can be taken and capturedduring the daytime to provide a meaningful comparison with daylightviews, the present invention can also be employed to enhance nightdriving by comparing memorialized images taken during the day whenvisibility is relatively good and playing back those images on asuitable split screen with real time images taken at night or duringinclement weather. The display of the recorded daylight single viewselected by current GPS position presents valuable information that maynot be visible in the current situation. The simultaneous display of thecurrent and previous condition would not be required. Sometimes ininclement weather or GPS obstruction, the current GPS could be lost. Bycomparison of current imagery to the database imagery, a current GPSposition could be selected by the closest image match to a specificframe in the database.

Because a driver may not have a direct view of his wheels inrelationship to a ravine or cliff or other hazard, an accidental rollover can occur. By using a GPS based offset that is some distance behinda vehicle, a navigation view can be presented to a driver. The outlineperimeter of the vehicle including its wheels can be superimposed overthe collected imagery. A view can be displayed that includes a view ofthe road and vehicle that appears to be from a camera behind thevehicle. This is accomplished by using a GPS position that isapproximately 20 feet from the current position to determine the videoframe best associated with the current position.

By placing a dot (Breadcrumb) in the proper location on a Cartesiancoordinate system on the display one can create a map that representsthe memorialized data. Breadcrumb marks on a top down map for each framecan then be composited to provide accurate maps of the driven area.These maps may be overlayed onto standard reference maps which may beimported into the system. This provides a graphic interface that can beused to present the view of the route that at any specific speedselected.

Change detection is enhanced at night by the use of auxiliary lightingin the visible or infrared wavebands. At night, using auxiliary lightingplaced lower than the camera, exaggerated elongated shadows produce anenhanced shadow effect to highlight changes. This auxiliary lighting canbe constant or pulsed to co-ordinate with the GPS capture of the frameafter a specific distance is traveled. The lighting can be in thevisible and or non-visible range.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic block diagram of the essential componentsnecessary in carrying out the present invention.

FIG. 2 depicts a typical split screen presentation of data which wouldbe presented to a vehicle operator in practicing the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Others, in the past, have taught methods of recording images andassociating them with GPS or geographic coordinate data to provideplayback that was position, rather than time dependent. For example,reference is made to U.S. Pat. No. 6,741,790, the disclosure of which isincorporated by reference. The '790 patent teaches a system includinghardware and software which are configurable and operable in threemodes, namely, recording mode, indexing mode and playback mode. In therecording mode, the hardware employed therein is connected to a GPSreceiver and video recording device and permits GPS data to be recordedon the same recording medium that images are recorded on using the videorecording device. As a result, a particular geographic locationassociated with particular images can be directly referenced by GPSdata. After recording the images, the hardware device is connected to acomputer having the appropriately installed software causing the GPSdata to be transferred from the recording medium to the computer as therecorded images are played. During this indexing step, the computerdraws a map showing all of the GPS reference locations recorded on therecording medium. Each map location represents a location where imageswere recorded with GPS data. Once the indexing mode is completed, thesystem disclosed in the '790 patent can be used to play back the imagesso recorded, in accordance with the selection of a location on the map.During this step, called the playback mode, a marker may be selected onthe computer generated index map and the system will cause the videorecording device to play back the images that were recorded at theselected location. The '790 patent also teaches recorded images beingtransferred to some other media such as a CD ROM or hard disk causingthe computer to play back the images that were recorded at that selectedlocation. However, no one to the applicant's knowledge has adapted thistechnology for the purpose presented herein. This new invention can usethe GPS to determine when a picture is recorded to reduce the number offrames in memory and which image to be called during playback. Althoughall frames can be recorded, it is best to only record those separated ata deliberately set distance. Thus the camera frame rate is dependent onthe velocity of the camera. This can also increase the required camerafrequency of recorded imagery past 30 frames per second for highertravel velocities through the use of a GPS triggered camera.

In addition, U.S. Pat. No. 6,895,126 teaches a system and method forsynthesizing images of a locale to generate a composite image thatprovides a panoramic view of the locale. The '126 patent disclosesemploying a video camera moving along a street recording images ofobjects along the street. The GPS receiver and inertial navigationsystem provide the position of the camera as the images are beingrecorded. The images are indexed with the positioned data provided bythe GPS receiver and inertial navigation system. According to one aspectof the '126 invention, an image recording device moves along a pathrecording images of objects along the path. As noted, the GPS receiverand/or inertial navigation system provides position information of theimage recording device as the images are being acquired. The image andposition information are provided to a computer to associate each imagewith the position information. As such, it is known in the prior art toassociate GPS coordinates with recorded image data for later playback.The disclosure of U.S. Pat. No. 6,895,126 is incorporated herein byreference.

As a first embodiment, the present invention involves a detection systemutilizing human intervention to detect changes in a scene by displayingvideo of a topography such as a roadway along with imagery captured froma previous trip. During acquisition, the system captures images from alive video feed and “meta-tags” them with additional informationincluding the location and orientation of a specific viewpoint.Important “meta tag” information would include, for example, directionof travel and GPS coordinates and/or elevation. These ‘meta tags’ arethen associated with specific frames of the previously taken videoimage. During playback, the system utilizes current location andorientation through GPS data acquisition to recall imagery previouslycaptured from approximately the same location and orientation. Ideally,the two images are displayed on a single screen above and below oneanother. In some applications, such as driving at night, only the storedview will be displayed. GPS data acquisition provides the coordinatesthat determine the closest “stored views” to the vehicle's currentlocation. This can also be done using various “closest matching frame”software. This system then selects corresponding views from a databaseto produce the nearly exact view that a vehicle operator, such as asoldier, is observing in real time in the vehicle. Compensation of thedatabase view can be made for variances in camera field of view, speed,distance to the center of the road and X and Y variances from theposition on the road from which the reference data was taken. Thesecorrections can be determined from physical sensors on the cameraplatform such as inclinometers and accelerometers or from calculationsderived from the sequence of images. The information is presented in anenhanced format for easy comparison to the observer to enable a vehicleoperator to make real time decisions about a road hazard.

As a preferred embodiment, the image can be deliberately distorted byincreasing the horizontal width to minimize the relative vertical motionor an unusual horizontal aspect ratio can be used to provide the viewerwith less vertical distance between before and after pictures thusminimizing eye movement This can be done for example by cropping theimage reducing the area above the horizon In carrying out this function,reference is made to FIG. 1 showing system 10 as constituting oneversion of the present invention. Specifically, system 10 includes acomputer 11 which is fed information from elements 12 and 13. Element 12is an image source generator such as a visual or thermal camera which isemployed in conjunction with GPS device 13 to “meta tag” framesgenerated from camera 12 and applied to computer 11 for storage. Theimage developed through this established reference would appear in thebottom portion of split screen 14 in image area 16 as discussed furtherin reference to FIG. 2. When traveling over the same route upon whichthe image projected in field 16 is played back, camera 12 againgenerates an image and the image thus created is coordinated with GPSdevice 13 and is again introduced to computer 11 which generates a realtime image in upper frame 15 of display 14. Because the stored image andcurrent image are both “meta tagged” with GPS coordinates, playback of aspecific scene in conjunction with a current route being undertaken arecoordinated for direct comparison.

The utility of the present invention can be appreciated in reference toFIG. 2. As noted, the split display screen 14 displays roadway 17 andits surrounding topography. The image displayed in lower half 16 ofdisplay 14 is an image taken of a roadway at an earlier date and time.Roadway 17 and its surrounding topography displayed in upper screenportion 15 shows a similar portion of the roadway, each view expandedhorizontally to enhance visual recognition of horizontally presentedobjects such as objects 19 sitting on the outside of shoulder 18. Imagesgenerated are ideally taken with a telephoto lens in order to provideadequate time for a vehicle operator to take action in the event thatthe operator wishing to investigate the nature of new objects 19 in theevent they represent munitions or other potential hazards applied to theroadway by hostile forces.

The image generated in row or frame 16, that is, that showing a previousdepiction of a roadway and surrounding topography can be acquired from amultitude of sources such as previous convoy runs, UAV, manned aircraftor robotics. Such information can be electronically submitted to acentral server which can be accessed by remote displays.

The main hardware components involved are a laptop PC, server or DVDburning system, small GPS receiver and a video camera, or other sensorproviding 2 or more dimensions of data. The principle softwarecomponents include simultaneous video/data capture and playback, thecapturing of location and orientation information from sensors on USBand/or serial ports, utilizing “meta tags” to associate the differentdata sets, storing the data in a format that can be shared or combinedwith other geo-spatial information, techniques for annotating the storedimagery to locate features such as potentially dangerous areas along theroute, an interface to interact with the data and a way to share thedata with others who will travel the same route at a later time.

An advanced functionality under consideration includes the ability totrack or predict viewpoint motion in cases of a limited GPS signal.

Other identified significant applications include training and dispatchor route planning. Driving simulators would be improved from the currentcartoon like images to real video.

Nighttime operations with IR and thermal sensors and alternative sourcessuch as magnetic maps or ladar can also be utilized.

A simple training device is a DVD of the recorded route that is playedon a DVD player.

As to simultaneous video capture and playback, live video will bedisplayed with previously captured imagery in an over/under fashion ontwo parts of the screen to facilitate change detection comparison. Tocreate a random accessible database (as opposed to an MPEG video stream)a JPEG storage architecture can be used. The capture of the still imageswill be based upon parameters including the distance traveled andheading.

Turning to capturing location and orientation information from sensorson USB and/or serial ports, the video, GPS and heading sensors willinterface to the system through the USB or serial ports and/or videoframe grabbers. A Panasonic® Toughbook® PC, for example, will hostsoftware to pull and retrieve information from the sensors on theseports.

“Meta tags” are used to associate the different data sets. “Meta tags”are a way to associate different data sets without requiring explicitrelationships to be exercised. This system will utilize “meta tags” torelate features including the location and orientation of a viewpointwith its corresponding image. Time of day and unusual occurrences canalso be “meta tagged”. It is part of the invention to include theheading information in the “meta tag” so that the direction of travelgoing north or south can determine the proper image. Other data can beelevation and annotations of unusual occurrences. Pre-determinedcommands can also be embedded in the data and are only displayed inproximity to annoted geo-referenced “meta tagged” markers.

Data is stored in a format that can be shared or combined with othergeo-spatial information. The system will store the data in a format thatcan be used with other geo-spatial products such as those routinely usedin the geo mapping industry with layers for features such as elevations,names of streets, rivers wireframes or shape files such as those fromESRI.

The present invention provides a way to share the data with others whowill travel the same route at a later time. A client server model (datafile management server) can be used to store and share the data amongstpatrons. Data can be transferred between the client and server usingnetworks or DVDs or other data media. Capture laptops will have DVDrecording systems that will be able to publish DVDs for distribution tothe server or directly to other users. The system design and topologywill depend on what type of network infrastructure and bandwidth isavailable in the theatre of operation. The server will have a top downmap view of the acquired vehicle paths. Any point on a path can be mouseclicked to provide the specific in vehicle view.

As an example, a PC was ruggedized to operate in military vehiclesalthough other competitive laptop computers could be employed. Webcamand video sources can be used including images having a 9-degree fieldof view for small objects seen at 100 meters or a wider field of viewfor curved streets or urban environments.

It is believed that the mind's ability to register non-conforming imagesis vastly underestimated. By presenting images on a single split screenone on top of the other enables a viewer to immediately recognizeirregularities and often times one can determine the relevance of suchdifferences while ignoring different camera positions, fields of view,contrast, shadows and color differences between the images. Othertechniques such as alternating the frames at various rates, superimposition, matting of features, or fusing of the imagery can also beused.

It is contemplated that a previously recorded view of a roadwayincluding surrounding topography could be projected and compared to areal time view of the same region can be projected on the windshield ofa moving vehicle as well. This may be useful for nighttime driving.However, a preferred embodiment is to provide a split screen wherebypreviously recorded video is compared with live video taken from avehicle moving along a roadside. By providing one scene over anothercoordinated through the use of GPS coordinates “meta tagged” within acomputing system, the vehicle operator can most readily detectdifferences between the “before and now” videos. The current embodimentrelies upon commercial off-the-shelf GPS technology to retrieve apreviously recorded reference view from a database that is nearlyidentical to the current “live” vehicle-mounted camera view. Again,ideally, both video views are simultaneously presented on a singledisplay for visual comparison. Such a technique presents the two viewsin a manner that significantly augments and enhances the operator'snatural ability to detect a “change” of new surface object.Semi-automated change detection software may also be employed tohighlight disparities between the views. Such software is availablecommercially. In another variation, the rapid presentation between thebefore and after pictures creates an animation with the changed parts ofthe picture appearing to blink on and off.

In contemplating software applications, it is noted that single framesare available as geo-referenced data frames, but are used as top downmap views. Several commercially available software packages are videobased, but they generally use an MPEG compression protocol noting thatonly the initial frame could be geo-referenced and easily selectablewithout excessive decoding. MPEG compression further requires additionalprocessing bandwidth and the utilization, again, of key frames. The useof key frames presents significant problems for accessing a specificframe within two points and would require significant databasemanagement tools. It was found that JPEG compression could be moreviable by eliminating the need for key frame reference and complex dataaccessing and data management tools.

The present invention has been described as using video or thermalcameras to produce images. Different combinations can also be used forthe before and current situation data sets such as a daylight videocolor camera for the acquisition and a black and white thermal sensorfor the current night time image. In using thermal cameras, the presentinvention can be utilized at night while providing the necessary imagerecognition and image differentiation presentations for carrying out thepresent invention from the daytime image. Imagery can also be computergenerated from wire frame and/or texture maps. Magnetic profiles,multi-spectral imagery, radar, sonar or ladar are also viable for use inpracticing the present invention. Road like video, thermal HD, or lowlight NTSC video could also be employed as viable video sources notingthat the key in practicing the present invention is the production ofgeo-referenced discreet raster based imagery.

It is contemplated that, as an embodiment, the data produced fromgeo-mapping a specific route can be stored in a format that can beshared or combined with other geo-spatial information. There have beengeo-spatial extensions created for databases such as Oracle® sourcedsoftware or ESRI to make database access easier. The formats forexchanging information have been developed by organizations such as thatavailable online at www.opengis.org. It is noted that the InternationalStandard ISO 19136, the Geographic Information-Geography Markup Language(GML) contains instructions to store, combine and/or relate informationwith other geo-spatially reference data sets. There is a featuresub-type that has a coverage function over a spatial domain such aspopulation and density, and an observation is considered to be somethinglike a photo or measurement, noted with a time and possible othergeneral values for the observation. The present invention intends toutilize the observation features as a template for data storage.Notations such as cautions can also be made that are referenced andstored with time captured and GPS location camera orientation noted asreferences. Noting that Open GIS specification includes portrayals ofgeographic information, map-type interfaces can be employed usingsymbols on suitable maps that display points of observation and thuswill allow others to retrieve and display information interfaces createdin the practice of the present invention.

Although the preferred embodiment in practicing the present inventioncontemplates a comparison between archival video taken from a vehicleand comparing it to real time video taken in substantially the sameorientation, that is, from a moving vehicle, alternative video sourcessuch as forward looking aerial and down looking recognizance can beemployed herein. Skewed transformation and cropping can be employed tocreate a forward-looking aerial view into something akin to a view takenfrom a land-based vehicle. In doing so, aerial imagery must be employedin conjunction with processing not needed for land based image capture.For example, ego vectors on the aerial camera platform such as pitch,roll and yaw must be accounted for. Alternatively, the vehicle view canbe transformed to look like a top down view which could then be used byautomated change detection systems in workstations using down lookingaerial video. This can be done using a line scanning approach to thesequence of frames. In alternative image manipulation techniques, oncamera sensors such as inclinometers and digital compass information canalso be tagged to a specific frame and used to calculate the morphingparameters to more closely register before and current imagery.

As noted previously, there are various hardware options available inpracticing the present invention. For example, one could use a simplelow cost web cam to provide the necessary imagery. Traditional camerassuch as palmcorders with image stabilization could also be usedeffectively. Thermal imagery can be employed. Low light intensifiedcameras and low light video cameras can also be used as well as imagesproduced by magnetic imagery, radar, sonar and rendered images fromdepth maps can additionally be employed. If analog video signals are tobe employed (NTSC), a frame grabber card with onboard JPEG compressionis desirable. This is all capable of being implemented into a personalcomputer. Alternatively, a video to USB or Analog to Digital convertercan be employed for input sources. A Garmin® GPS device can be employedfor geo referencing including the GPS coordinates and time of day or aninertial GPS system could be used.

It is also considered an aspect of the present invention to provide, onan as needed basis, various indicators of potential hazards on the videoor appended audio archive. For example, voice commands, magneticsignatures, environmental sound, vehicle velocity, weather conditions,lighting and other environmental factors could be installed onto thememorialized depiction of a scene. On screen warnings of upcomingevents, checkpoints, or decision points could be triggered by GPSpositioning. In addition, vehicles can communicate with one anotherusing DVD and server laptop data links as well as file managementservers.

The software to be employed in practice in the invention must havecertain obvious requirements. It must be capable of processing a livecamera and sensor input while displaying output on a split screen.Although not required, the simultaneous recording and playbackcapability provides a system to automatically present the most recentdata. The software must be capable of receiving GPS input for currentvehicle positioning while selecting from a reference database of viewsthe one view designated by closest GPS coordinates. Although notrequired, the same GPS coordinates can be employed to shift X and Ycoordinates of the rendered view while displaying roadside threats,preferably in stereoscopic 3D based upon monoscopic 2D input signals.Better registration between the recorded and current image can beachieved using known techniques that consider the current camera egomotion and the acquired ego motion acquired from image interpolation oronboard sensors such as inclinometers, accelerometers or a digitalcompass. This onboard sensor information can be part of the meta-datafields embedded with each acquired image. This later feature isdesirable for in depth examination. Again, as noted previously, as apreferred embodiment, the software can also utilize existingsemi-automated change detection software to highlight on screensuspected hazards. The software code would utilize on board GPS devicesin real time culled from an appropriate database of the most closelystored video frame. The closest GPS view is selected from the databaseas the onboard camera simultaneously displays both images. The imagesare ideally expanded horizontally (or squeezed vertically) to make thenecessary X, Y adjustments based on road position and/or onboard sensorswhile recording images from onboard cameras to a storage device forfuture reference. Morphing software can also be used to improve theregistration. It is also contemplated that the present invention willtrack the time that the video was captured and elevation and stored asthis information as part of the “meta tag” data set. An XML schema willprovide an interface for such information.

As noted previously, real time change detection software can be employedin conjunction with the present invention. Standard Geo Referenced Data(ARC Info/GIS) information can be imported and exported. In additionchange detection software can compliment the system. Use of suchsoftware can be of assistance to eliminate false positives thus makingthe present invention more viable to an untrained operator. Images canbe displayed to the operator on HMDs, heads down displays, LCDs,projection and laptop screens.

Although the present invention has been fundamentally described in termsof its use as an aid in high-risk applications, the invention can alsobe used in consumer and industrial environments as well. For example,rural and suburban areas can be mapped and “meta tagged” with a GPScoordinates and stored in an appropriate library. In the event of anatural disaster, such as an earthquake where roadway surfaces andsurrounding topography can be substantially altered and thus create ahazard, such alteration could be readily visualized through the use ofthe present invention.

Furthermore, a downlooking camera from a submarine or surface watercraftcan be used to precisely navigate a channel and avoid hazards. It canalso be used to better land an aircraft in unfamiliar terrain or enter aport and make precise turns in relation to visual objects such as buoys.Comparisons of objects such as power lines, road markers and paintedlines, roadside vegetation, tunnels and movement of terrain can also bemade. Use of this invention as a navigational aid during nighttime, orfor documenting deliveries of equipment along a route can also be made.Precise positioning of a vehicle can be made utilizing the comparison ofthe perfect position previously recorded to the current position. Forexample determining when to turn into a channel can be precisely made bymatching shoreline trees, docks, buoy positions previously recorded byan expert navigator can be compared to the current position. This sameprincipal can be used by large trucks trying to enter narrow loadingdocks.

1. A method of storing geo-referenced imagery, the method comprising:providing a system, wherein the system comprises a computer processorand a database; receiving, by the computer processor, a first set ofimage data captured at a first rate and a first set of meta-tagsassociated with a field of view at a location along a travel route; andstoring within the database at second rate a second set of image dataand a second set of meta-tags, wherein the second rate is independent ofthe first rate and based in part on a distance traveled along the travelroute.
 2. The method of claim 1, wherein the first set of image datacomprises the field of view from an image acquisition device associatedwith a vehicle.
 3. The method of claim 2, wherein the vehicle is anunmanned aerial vehicle, a manned aircraft, a manned vehicle, or arobotically-controlled vehicle.
 4. The method of claim 2, wherein thefirst set of image data is captured from the image acquisition device ata frame rate dependent on a velocity of the image acquisition device. 5.The method of claim 2, wherein the image acquisition device is one of avideo or a thermal camera.
 6. The method of claim 1, wherein the firstset of image data is video data.
 7. The method of claim 1, wherein thesecond set of image data is sampled from the first set of image data andthe second set of meta-tags is sampled from the first set of meta-tags.8. The method of claim 1, wherein each image within the second set ofimage data is associated with one or more meta-tags from the second setof meta-tags based in part on geo-location data.
 9. The method of claim1, wherein the second set of image data is stored as one or moreindividual images.
 10. The method of claim 9, wherein the one or moreindividual images are stored in a JPEG format.
 11. The method of claim 9wherein the one or more individual images are sampled from real-timevideo data.
 12. The method of claim 1, wherein the first or the secondset of image data is discrete raster-based imagery based on one or moreof video imagery, computer-generated imagery, multi-spectral imagery,thermal imagery, magnetic imagery, radar, ladar, or sonar.
 13. Themethod of claim 1, wherein the first set or the second set of meta-tagsincludes one or more of GPS coordinates, elevation, or headinginformation associated with the field of view provided by an imageacquisition device.
 14. The method of claim 1, wherein the first set orthe second set of meta-tags includes data received from one or more ofinertial sensors, inclinometers, accelerometers, digital compasses, ortiming devices.
 15. The method of claim 1, wherein the first set or thesecond set of meta-tags includes information representing a velocity ofan image acquisition device capturing the image data.
 16. The method ofclaim 1, wherein the first set or the second set of meta-tags includestime of day information.
 17. A method of compressing geo-referencedimagery, the method comprising: receiving, over an interface, a firstset of image data captured at a first rate and a first set of meta-tagsassociated with a field of view at a location along a travel route;assigning, using a processor, one or more meta-tags from the first setof meta-tags to each image from the first set of image data based ongeo-location data; and storing within a database at second rate a secondset of image data and a second set of meta-tags sampled from the firstset of image data and the first set of meta-tags, wherein the secondrate is independent of the first rate and based in part on a distancetraveled along the travel route.
 18. The method of claim 17, wherein thefirst set of image data is captured from the image acquisition device ata frame rate dependent on a velocity of the image acquisition device.19. The method of claim 17, wherein the second set of image data isstored as one or more individual images.